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Abstract 

For many networks, the connection pattern (often called the topology) can 
vary in time, depending on the changing state of the modules within the net¬ 
work. This paper addresses the issue of nesting such mode-dependent net¬ 
works, in which a local network can be abstracted as a single module in a larger 
network. Each module in the network represents a dynamical system, whose be¬ 
havior includes repeatedly updating its communicative mode, and these mode 
in turn dictate the connection pattern. It is in this way that the dynamics of the 
modules controls the topology of the networks at all levels. This paper provides 
a formal semantics, using the category-theoretic framework of operads and their 
algebras, to capture the nesting property and dynamics of mode-dependent net¬ 
works. We provide a detailed running example to ground the mathematics. 


1 Introduction 

If we find that a story or structure repeats itself at various scales of a model, it 
is often a useful exercise to formalize the model using operads, because doing so 
constrains the model to be highly self-consistent. There is an operad of complete 
sentences: "Here is a sentence, here is another sentence, and this entire quotation 
is a sentence." (There is (or may (also) be) an operad of parentheticals.) More 
importantly, there is an operad of networks, one that naturally models networks of 
networks. 

To say that there is an operad of X is to say that there is a hierarchical, modular 
theory of X. In previous work [21, 25, 28], operads that describe special sorts of net¬ 
works, called wiring diagrams, were given and shown to model databases relations, 
digital circuits, and open continuous-time dynamical systems, among other exam¬ 
ples. A wiring diagram is a fixed, graph-like arrangement of nodes and directed 
edges where (1) nodes represent boxes CD with differentiated input and output 
ports and (2) one can "cut-and-paste" graphs into or out of other graphs by expand¬ 
ing or contracting at a given node. In this note, we extend wiring diagrams to the 
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case where the network topology is not fixed in time but instead varies with respect 
to the states of various nodes in the wiring diagram. The states of the nodes define 
a local "mode", so we have dubbed these diagrams mode-dependent networks. 

Motivating example. Many examples of dynamical processes on complex net¬ 
works come from biology and living systems, and in such systems the network 
topology often changes over time as the state evolves; changes in topology in turn 
influence the state. One classic example is a (human) neural network, where connec¬ 
tions are constantly formed or pruned as neurons fire. We present here a network 
model of the visual system that composes seamlessly from the firing dynamics of 
individual neurons up to the large-scale dynamics of brain models that simulate 
blinking, light adaptation, and visual processing. 1 

Example 1.1. Consider the diagram below. 



Figure 1: A discrete dynamical system of the visual pathway, interpreted over a 
mode-dependent network in Cmdn, the operad of mode-dependent networks. The 
thicker wires represent many parallel connections. 


The visual pathway from retina to cortex is a complex neurological system that 
combines autonomic behaviors (blinking, saccading) with layers of image process¬ 
ing in the visual cortex. In this diagram, Ej and E 2 represent the eyes and eyelids, 
P represents a collection of nuclei in the brainstem that regulate the corneal (blink) 
reflex, and V represents (a slice of) the visual cortex. VS represents the entire visual 
system. This setup can be described using an operad because the outer box is of the 
same nature as the inner boxes—i.e., the visual pathway is itself a neural network 
which may be placed within a larger neural network—and the process of assembly 
can repeat ad infinitum. In this way, one can recursively build up networks of net¬ 
works. The goal of this paper is to show how operads can model not only networks 
of networks but also complex, heterogeneous dynamics over such networks. 

The mode-dependency of Figure 1 may not be obvious at first glance. We will see 
it in later examples. This follows a common motif in networked systems; changes 

1 There is an ulterior motive for this example; we believe that operads are useful not only in the 
analysis of complex networks but also in the design of complex (scientific) models, such as for large- 
scale brain models. This example is a demonstration of both uses. 
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in connection pattern are not typically visible in a macro analysis of networks of 
networks, yet small changes in connectivity can drastically change the dynamics of 
the system as a whole, for example in power grids [8], the Internet [1], and functional 
brain networks [20]. To see the structure, one must zoom in. 

Similar stories of modularity and hierarchy have been considered before, includ¬ 
ing networks of networks [20], multilayer networks [16, 8], and tensor representa¬ 
tions thereof. Operads can be used to enrich these graph-theoretic models; operads 
provide a formula for zooming in and out of a complex network, and the proofs 
in this paper show that the formula for rewiring and assembly is consistent and 
independent of one's choice of how and where to zoom. This requirement puts 
fairly strict constraints on the formalism, as mentioned above. Not any seemingly- 
workable definition of mode-dependent dynamical systems will actually satisfy the 
nesting property. 

All this will be made formal below. We refer the reader to Mac Lane [18], Awodey 
[2], or Spivak [26] for background on category theory (in decreasing order of diffi¬ 
culty), to [17] for specific background on operads, algebras, and monoidal categories, 
and to [11], [19], or [16] for background on complex networks and multilayer net¬ 
works. Other category-theoretic approaches to networks and their dynamics include 
[9, 10], [4, 5], and [23], 

In Section 2 we will cover the prerequisites in category theory, before giving the 
precise definition of mode-dependent networks in Section 3 and of modal dynamical 
systems in Section 4. Readers interested mostly in applications may skip directly 
to Section 5, though we recommend at least skimming some of the examples in 
Section 4. 

2 Background 

We begin with some notation and basic terms from category theory. 

Notation. Let Set denote the category of sets and functions between them. Let 
FinSet C Set denote the full subcategory spanned by the finite sets. If A,B G Ob A 
are objects of a category, we may denote the set of morphisms between them either 
by Hom^(A, B) or by 7?(/l, B). If A G Ob 7? is an object, we may denote the identity 
morphism on A either by id /\ or simply by A. If there is a unique element in ( A , B), 
we may denote it !: A — > B. For example, there is a unique function !: 0 — » A for 
any set A. The symbol 0 represents the empty set. For any category, let Mor 0' 
be the union of all the hom-sets, MorA = L-U^g^Hom (A, B). There are maps 
dom, cod : Mor 0 —> Ob 0 sending a morphism / to its domain and codomain, 
respectively. 
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Recall that wiring diagrams are made up of boxes with a notion of input and 
output port, each labeled by the type of data that goes through that port. The 
following definitions are used to make this precise. 

Definition 2.1. The category of typed finite sets, denoted TFS, is defined as follows. 
An object in TFS is a finite set of sets, 

Ob TFS := {(A, r) | A € Ob FinSet, r: A —> Set)}. 

We call r the typing function, and for any element a G A, we call the set r (a) its type. 
If the typing function r is clear from context, we may abuse notation and denote 
(A, r) simply by A. 

A morphism q: (A, r) —> (A', t') in TFS consists of a function q: A —t A' that 
makes the following diagram of finite sets commute: 



Set 


We refer to the morphisms of TFS as typed functions. The category TFS has a 
monoidal structure (A, r) U (A', r'), given by disjoint union of underlying sets and 
the induced function AUA'^ Set. 

Definition 2.2. Given a finite set A and a function r : A —t TFS, we denote by 
(A, t) = A, the cartesian product 

A := nrW. 

aeA 

We call the set A the dependent product of A. 

Taking dependent products is a functor TFS op —> Set, i.e., a morphism q: A —» A' 
induces a function q: A' —> A. In fact this functor is strong in the sense that if A and 
B are finite sets, then there is an isomorphism, 

AaJB ^ A x B. (1) 

Example 2.3. We give three examples of typed finite sets and their dependent prod¬ 
ucts. 

1. If A = {1,2,3} and r: A —> TFS is given by r(l) = r(2) = r(3) = Z, then the 
dependent product is (A, r) = Z x Z x Z. 

2. Let {*} be an arbitrary one-element set. Consider the typed finite set {*} A 
TFS, sending * to the set r(*) = {a, b,... ,z}. Then the dependent product is 
simply ({*}/t) = {a ,b,...,z}. 

3. Consider the unique function !: 0 —» TFS. Its dependent product is 0 = {*}, 
because the empty product is the singleton set. 
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Remark 2.4. In this paper, we choose to speak in terms of operads rather than 
monoidal categories since the overarching idea of modularity—one thing built from 
many, many small boxes wired into one large box—is operadic. That said, the formal 
mathematics will generally be written using the framework of symmetric monoidal 
categories since doing so avoids the use of subscripts in our notation. We are im¬ 
plicitly referring to a functor 

G\ SMC -G Oprd 

from the category of symmetric monoidal categories and lax functors to the category 
of operads—by which we mean symmetric colored operads—and operad functors. 
If (‘if, 0) is a symmetric monoidal category then the operad has the same objects 
as 'if, and a morphism (Xi, ..., X„) —>■ Y in is defined as a morphism X\ 0 • • • 0 
X„ —* Y in c if. See [17, Example 2.1.3]. 

Note that monoidal categories are used in [4] and [5] to study networks, and 
some confusion may arise in comparing their work to our own, unless care is taken. 
See [27]. One way to see the difference is that we are focusing on the fact that 
networks nest, i.e., that multiple dynamical systems can be gathered into a network 
that is itself a single dynamical system. In this way, our work more closely follows 
the intention of [7] or [12]. However, neither of these references uses category theory, 
though the latter mentions it as a plausible approach. 


3 Mode-dependent networks 

Definition 3.1. A box is a pair X = (X m , X out ), where X m , X out G ObTFS are typed 
finite sets. Each element of X m LI X out will be called a port. 

If X and Y are boxes, we define a wiring diagram, denoted q): X —> Y, to be a pair 
of typed functions cp = ( (p"\ <p out ): 

<p in : X in —> Y in U X out (2) 


We define the composition formula for wiring diagrams cp: X —» Y and ip: Y —» 
Z as the dotted arrows below, the indicated compositions in TFS: 


X m ---> z m u x out 

'f' 

yin z in U(p out ux out 

y in u X out - > Z in y yout y X out 

!/> ta UX out 



( 3 ) 


This defines a category, which we call the category of wiring diagrams, and denote 
WD. It has boxes as objects and wiring diagrams as morphisms. It has a symmetric 
monoidal structure defined by disjoint union, which we denote by U: WD x WD —> 
WD. We denote the operad underlying WD by ^WD- 


5 







Remark 3.2. By convention, we label the input ports of a box P by Pf',Pf x ,... and 
the output ports by P° ut , P^ ut ,.... We also often suppress the typing r. For a zero- 
input, two-output box P = ((0,!), ({P° ut ,P£ ut },r)), where r(P° ut ) = r (P£ ut ) = L, 
we would write P m = 0 and P out = {P° ut , P° ut }, and we might draw it in any of the 
following ways: 


p 

P° ut :L 

P 

pout 

p 

_ 

P° ut :L 


■pout 



Example 3.3. Consider a wiring diagram with three boxes A\, Ai, A 3 wired into a 
bigger box, B. We can define a morphism cp: (A|, A 2 , A 3 ) —> R in A'wd by spec¬ 
ifying equations such as cp m (Af a ) = A^, cp out (B° ut ) = an d so on - (This is 

equivalent to a morphism cp : X —> B in WD where we use the symmetric monoidal 
product to define X = A\ U A 2 U A 3 .) Specifying a diagram is equivalent to specify¬ 
ing the two functions cp m and (p out . Note that the connections in a wiring diagram 


cp: (A 1 M 2 , A 3 ) —> B 



Figure 2: A wiring diagram. 


are all n-to-one for some n, e.g. wires may naturally "split" but may not combine 
unless through a box. Every output node maps to at most one input node, but there 
may be many output nodes for one input node. See also [28, Definition 3.5]. 

By now we can define static networks, e.g. pictures. To define networks that can 
morph and change, we give the following definition: 

Definition 3.4. We define a symmetric monoidal category, called the category of 
mode-dependent networks and denoted MDN, as follows. An object in MDN, called a 
modal box, is a pair (M, X), where M E Set is a set and X: M —> Ob WD is function, 
sending each mode to a box (see Definition 3.1). We call M the set of communicative 
modes, or just mode set for short, and we call X the interface function. If X factors 
through a one-element set {*}, we call (M,X) a mode-independent box. 

We define Hom MDN ((M, X), ( N,Y )) to be the set of pairs (e,a), where e : M —> 
Mor WD and a : M —> N are functions making the following diagram commute: 


M = 
x 

> r 

Ob WD <■ 


dom 


M 


Mor WD 


-> N 


cod 


Y 


4 Ob WD 


(4) 
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We call a morphism (e,cr) € HomMDN((M, X), (N, Y)) a mode-dependent network , and 
we call e the event map, following [11]. We can also write 

e E n WD(X(m),y(m)). 

mgM 

In the special case that X, Y, and e factor through the one-element set {*}, i.e., if 
neither the shape of the boxes nor the wiring diagram change with the mode, then 
we say the network is mode-independent. 

Given two composable morphisms (mode-dependent networks) 


M 0 

x 0 


Mo 

e 0 


^0 


-4 Ml = 
Xi 


Mi 

e\ 


0i 


-4 M 2 

x 2 


Ob WD <-— MorWD — 4 ObWD f — MorWD —-4 ObWD 


dom 


cod 


dom 


cod 


the composition formula is given by {e\, <J\) o MDN (eo/ Oo) := (e, a), where a := CT\ o cr 0 
is the composition of functions, and where e: Mo —4 MorWD is given by composi¬ 
tion in WD: 


e(m 0 ) '■= ei cr 0 (m 0 ) o e 0 (m 0 ). (5) 

The monoidal structure on MDN is given on objects by 

(M,X)®(N,Y):=(MxN,XUY), (6) 

and similarly on morphisms. Here, X U Y is shorthand for the composite 
MxN ObWD x ObWD 4 ObWD. 

The unit object in MDN is ({*}, 0 ), where {*} is the singleton set, and 0 is the 
monoidal unit of WD. 

Remark 3.5. In [11], attention is paid to the image of the event map e: M —4 WD(X, Y), 
which is denoted sY C WD(X, Y). At certain points in that discussion, sY is chosen 
independently of e, but at others it is assumed to be the image of e, see [11, Remark 
4.10(2)]. In Definition 3.4, we could have defined a morphism in MDN to consist of 
a triple {sY,e,cr), where sY C WD(X, Y) and e: M —4 sY. In this case, composition 
would also involve composing the various sY' s, but this is straightforward. How¬ 
ever, specifying sY independently seemed superfluous here, especially given Field's 
remark. 

Example 3.6 (Building a basic model). The human retina is made up of light-sensitive 
nerve cells which convert any image projected onto the retinal detectors (rods and 
cones) into neural signals. Like almost all neurons, retinal nerve cells have three 
physiological modes: polarized (the neuron is inactive), depolarized (the neuron is 
active and firing), and hyperpolarized (the neuron has just fired and is not receiving 
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cp:N(m^)—^N(*) 


il’:N(mn)—>N(*) 


nNo,i] 



- AT- 

R 


IV 



N° ut :{0,l} N“: [0,1] 


AT - 

R 




N° ut :{0,l} 


n;^=polarized 


m;v=depolarized, hyperpolarized 


Figure 3: Three modes of a retinal nerve, represented by the wiring diagrams cp, i p. 


input). In the retina the situation is somewhat more complicated; there is an extra 
mechanism that tracks the cell's adaptation to background luminance [14]. This is 
modeled by the two wiring diagrams, cp and ip, in Figure 3. 

The modal box (Mn, N ) represents the nucleus of a retinal nerve cell which takes 
a value in [0,1] as input and outputs a spike in {0,1}. The outer box (Mr, R) is a sim¬ 
ply a container for (Mn, N). The different modes Mn = {polarized, depolarized, hyperpolarized) 
each specify a (normal) box in Ob WD: 

N( depolarized) = -Q- = (N”,N out ) = ({N“ N£*}, {N° ut ,N£ ut }) 

N (hyperpolarized) = CO = (N”,N 0Ut ) = ({Nf,N^}, {N° ut ,N fc out }) 

N(polarized) =C3= (N in ,N out ) = ({N' b n }, {N° ut ,N^ ut }) 


The box (Mr, R) = R = £3 is mode-independent, so we represent its mode set 
by Mr = {*}. The entire mode-dependent network in Figure 3 is defined by a 
morphism Nerve = (e, cn) : (Mn,N) —» (Mr, R) in MDN with the following data: 

e(depolarized) = e (hyperpolarized) = cp E MorWD 
where cp in (N^) = Rf, 

-AN™) = N° ut , 

r‘(C) = K ut 

e(polarized) = ip & MorWD 
where i/> in (N^) - N fc out 
ip out (R° ut ) = N° ut 

^(polarized) = cr\ (depolarized) = ^(hyperpolarized) = * E Mr 

One then checks that the data satisfies the commutative diagram in the definition 
of a mode-dependent network, e.g. Definition 3.4. 

Example 3.7 (Forming products and compositions). We can form a simple model 
of the eye by taking the (symmetric, monoidal) product of all the nerve cells, i.e. 
E = Ri U R 2 U ... U R n . (Note, (Mr, E ) has mode set Mr = n^R, — {*}•) To define 
a mode-dependent network on E we then specify a morphism 

Eye = (e,(T[) : ((M Rl ,Ri), ..., (Mr„, R„)) —>■ (Mr,E). 
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This morphism lives in the operad &uoUr so composing it with another morphism 
means "zooming in" on the diagram, while precomposing means "zooming out". 
For example, to display the interiors of each R„ we precompose the mode-dependent 
network Eye : ((Mr 1 , R\), ..., (Mr 3 , Rf)) — t (Me, E) with three copies of Nerve : 
(Mn, N ) —> (Mr, R), as in Figure 4. 



-AT - 

Ri . 





u 




AT . 

r 2 


IN/ 2 



u 



.AT~. 

£3 


m 3 



Figure 4: The monoidal product of multiple mode-dependent networks, displayed 
with an instance of the mode-dependent network Eye o (Nerve, Nerve, Nerve). 


Proposition 3.8. Every iviring diagram is canonically a mode-independent network in the 
follozving sense: There is a strong monoidal functor 

I: WD ->■ MDN 

given by I(X) := ({*},X), where {*} is any choice of singleton set. 

Proof This is straightforward. A wiring diagram cp G WD(X, Y) is sent to I(cp) := 
e MDN(7(X), 7(Y)), where e (p : {*} — > WD(X,Y) picks out cp, and {*} 
is the identity function on the singleton set. This network is mode-independent 
because it has only one mode. It is easy to check that 7 is a strong monoidal functor. 

□ 

Given any mode-dependent network (e,cr): (M, X) — > ( N,Y ) in MDN, it is often 
convenient to factor e G Y[meM(X(m),Y(m)) into two parts, 

e in G n TFS(X hl (m),Y in (m) UX out (m)) 

m6M 

e out G n TF s (Y out (m),X out (w))- 

m6M 

These are obtained simply by considering the two components in (2). Composing 
with the dependent product functor TFS op —> Set from Definition 2.1, we obtain the 
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following dependent functions: 


e in G Set(Y in ()n) x X 0Ut (?7z),X in (m)) 

m£M 

G Set(X out (;n), Y°^(m)). 

m£M 

For any communicative mode m G M, these functions specify how information 
will travel within the network (e in ) and how information will be exported from the 
network (e out ). 

4 Dynamical systems on mode-dependent networks 

The goal of this paper is to define an algebra, i.e., a lax functor V: MDN —>■ Set, of 
(synchronous) discrete dynamical systems on a mode-dependent network. Below, 
we will supply the data that defines V in Definition 4.1 and then prove that it satisfies 
the conditions of being an algebra in Proposition 4.4. 

Let (M, X) G Ob MDN be a modal box, where M G Set is a mode set and X = 
(X m , X out ): M —» ObWD is a modal box. Here X m , X out : M —> TFS. Recall from 
Definiton 2.1 the notation X m , X out : M —> Set for their dependent products. We 
define the set V{M,X) as 



/ 

S G Set, q: S ->• M, 1 

V{M,X) \= < 

(S, q,f in ,f out ) 

f m eV[Set(x^(q(s)),s), 1 

seS | 



/cut g nx° ut (<7( S )) 


< 

seS ) 

is, an element 

of V{M,X) is a 

4-tuple consisting of: 


• a set S G Set, called the state set; 

• a function q: S —>■ M, called the underlying mode function; 

• for each s G S, with underlying mode m = q(s), a function / m (s) : X m (m) —> S, 
called the state update function; and 

• for each s G S, with underlying mode m = q(s), an element / out G X out (m), 
called the readout. 

We may denote an element of V(M, X) simply by (S, q,f ), and we call / = (/ m ,/ out ) 
an open dynamical system with state set S, following [28]. The whole 3-tuple ( S,q,f ) 
will be called a modal dynamical systejn. 

For the lax monoidal structure, one coherence map, V(M,X) x V(N, Y) —»■ 
V(M x N, X U Y), is given by cartesian products 

SxxY : = Sx x Sy 

<7xxY : = qx x qy (8) 

fxxY '■= fx X /y 
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Note the isomorphism (1). The other coherence map, {*} —> Vi{*} ,&), is the 
element ({*},!, !, !), where each ! denotes the unique morphism of the evident type. 

Given a morphism (e,cr): (M, X) —> ( N,Y ) in MDN, we need to provide a 
function V(e,cr): V(M, X) —y V{N,Y). For an arbitrary modal dynamic system 
( S,q,f ) G V(M, X), we define 

V{e,cr)(S,q,f) := (S,r,g) (9) 

where S is unchanged, r = a o q is the composite S M N, and for every s G S 
the update function g m (s) and the readout g out (s) are given as follows. Let m = q(s), 
so we have e in (m): Y in (m) x X out (;n) —> X in (m) and e out (m): X out (m) —^ Y out (m). 
Then the readout is given by 

g out (s) = i°ut(m)(/°ut(s)) (10) 

and the update y' n (s) is given by the following composition in Set: 

W(m) Ym( '” )x/ (b) > Y^{m) x X out (m) 

l 
l 

r (s) i 

i 
i 

S 4-:- 

/ m (s) 

We can restate this (still with m = q(s)) as follows: 

r(s)(y) = r(s)l^(m)(y,r\s))) (H> 

Definition 4.1. We define V as the data (7), (8), and (9) above. So far they are only 
data; we will show that they constitute a lax monoidal functor V: MDN —> Set in 
Proposition 4.4. 

Remark 4.2. If (M, X) is a mode-independent box, we can write X = X(m ) for all 
m G M at which point we can rewrite the state update and readout as functions 
f m : S x X^* —» S and / out : S -4 X°W 

Here is the intuitive idea of how information flows through a mode-dependent 
network. It begins with each inner box X,- converting its state to an output. These 
outputs, together with the input to Y, are carried through the wires and either 
exported from Y or fed as inputs to the X ; . These inner boxes then use this input to 
update their state, and hence their communicative mode, and hence their own shape 
and that of the network. The process repeats. 

Example 4.3. Before proving the main result, we motivate it by defining a modal 
dynamical system (Sn, qn,/ n) £ ?(Mn, N) on our simple model of the retinal 
nerve (Mn, N) in Example 3.6. For convenience, we reproduce the diagram from 
Figure 3. 


e m (m) 

X in (m) 
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cp:N(m N )-±R(*) 




Rjf: [0,1] 



.AT. 

R 


JM 



Rr‘:{04} Rif:[0,1] t 



A 7 - 

1 R 


N 



R° ut :{0,l} 


n!fv/=polarized 


n!fv/=depolarized, hyperpolarized 


State sets. Let S ^ x A, where A = K+ is the adaptation of the cell to light— 

the more adapted to light the cell is, the more light needed to activate the cell. 
The wrapper R has state set equal to the product of its internal boxes, in this 
case just N, so Sr = S^. Mr = {*}. 

Underlying mode function. We define qjsi : Sjv —t Mn to be the first projection. 
qR : Sr —>• Mr is trivial, since Mr = {*}. 

State update function. Define two constants, a firing threshold a E [0,1] and an 
adaptation constant (6 > 1. For m = polarized, let : S,v x [0,1] —> S be 
defined by 


(depolarized, a + x/R) x — a > a 

f§(rn,a,x) = < 

I (polarized, a Ip) otherwise 

In other words, the cell depolarizes ("fires") from its inactive state if the input is 
above a certain threshold a, and simultaneously increases its stored adaptation 
parameter a ^ a + x/ft. 

If the cell is either its depolarized or hyperpolarized modes, then there is no 
separate input for light received, so the update function /“ depends only on 
the previous state, e.g. 


/nKa) 


(hyperpolarized, a/j 6 ) m = depolarized 
(polarized, a/ft) m = hyperpolarized 


The state update function for the wrapper (Mr, R) is just the same as that for 

(M n ,N). 


Readout. In this case we can define the same readout function /] 
for all modes: 

( 

1 m = polarized 
0 otherwise 


out 

N 




{ 0 , 1 } 


f°N ('«/«) = 


/} ut is defined identically. 

The point of calling V : MDN —> Set an algebra (i.e. a functor) is that one can 
construct new modal dynamical systems using essentially algebraic operations. For 
example, let 

Eye' = Eyeo (Nerve,..., Nerve) 
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be the mode-dependent network of the "eyeball" we showed in Figure 4. Then we 
can obtain a new modal dynamical system (, Se, CjE,fE ) £ V{Me,E) by applying a 
function 

P(Eye') :V((M Ni/ N 1 )^(M N2/ N 2 )0...))^V(Me,E) 
to (Sn, <7n//n) as defined above. Specifying 

(Se^eJe) = 'P{Eye')((S Nu q Nl ,f Nl ),-, (S Nk ,q Nk ,f Nk )) 

is fairly straightforward: we first obtain an intermediate modal dynamical system 
on (Me, E)—think of this as the most 'general' dynamical system we can place on 
(Me, E)—by using the given coherence map ( 8 ) to define a dynamical system with 
the following data 


Se — S Nj x Sn 2 x ... x S Nk 
<?E = CjNi x <?N 2 X ... x q Nk 

fin __ fin v fin v v fin 

JE JNi J N2 *’* 

rout _ rout v rout v v rout 

JE J Ni J A/2 * * * J Nk 

We can then define a new dynamical system by defining a map on this dynamical 
system according to Equations (10) and (11). 

Since Eye' here is just the disjoint union of the several retinal nerves (wrapped 
up in wrappers Rj), this intermediate modal dynamical system is just what we want. 

Proposition 4.4. With the data given in Definition 4.1, the map V: MDN —> Set is a lax 
monoidal functor. 

Proof. It is clear that the coherence maps satisfy the necessary unitality and associa¬ 
tivity properties, because they are just given by cartesian products. So it remains to 
show that V is functorial, i.e., that it commutes with composition. 

Suppose given morphisms 

(M 0 ,Xo) (Mi,Xi) (M 2 ,X 2 ) 

in MDN, and let (e,a\ o no) = (ei,cq) o (eoWo) be the composite as in (5). Suppose 
that (S,qo,fo) £ 'P(A4o, Xq) is an arbitrary element. For notational convenience, 
define 


(S,< 7 i,/ 1 ) : = 'P(eo,iio)(S,qo,ff), 

(S, q 2 r fi) ■= V{e 1 ,cr 1 )(S,q 1/ f 1 ) / 

( S,q 2 >,f2 ') : = V(e,(Ti oob)(S,q Q ,fo), 

see (9). We want to show that (S, q 2 ,fi) = ? (S,c/ 2 ',/ 2 ')- It is obvious that 


qi = qi’ = cr\ o ctq o q 0 , 
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so it only remains to show that f 2 = ? f 2 i, i.e., that = ? and ut = ? f^ 3 *. 

We begin by choosing any s E S, and we let ntQ = cjo(s), m\ = q i(s) = ao(qo(s)), 
and m 2 = qi(s). For the first desired equation, we are trying to prove the equality 
/ 2 m (s) = ? ff(s) between functions X'Ximi) —? S. Thus we choose an element X 2 E 
^ 2 n (?H 2 ) and, using (10) and (11), we have: 

/f(s)(x 2 )=/f'(s)(T(mi)(^/T" 1 (s))) 

=/o n <s)(4 ; ("io)(T('»i)(®,/r u '(s))-/r(s))) 

= /o”’(s)uS'(m 0 )^ef(m 1 )(»,eS u, (mi))(/J u, (s))),/J u *(s)^j 

= fS'(s)(^(m,)(x 1 ,fr(s))) =/P(s)(x 2 ) 

For the second desired equation, we are trying to prove the equality f 2 ut (s) =? 
f%X l {s) between elements of X^ 1 . Again using (10), we have: 

/ 2 ° ut (s)=^r( mi )(/r(s)) 

= £ i ut ( m i) ( e o ut (« z o) (/o° ut (s))) 

= i^(m 0 )(/ 0 out (s)) =/ 2 °, ut (s). 

This concludes the proof. □ 


5 Applications 

In principle, operads are general enough that they can be applied in any instance 
where nested graphs, nested networks, and nested graphical models are used. Com¬ 
position in the operad immediately gives the semantics of networks of networks, 
along with a consistent and procedural way to plug different networks into each 
other. Mode-dependency models the additional interaction between dynamics and 
topology. The formalism is well-suited to heterogenous graphical models of dynam¬ 
ical systems with many functional dependencies between variables [28], but it can 
also be adapted to more homogeneous models where all ports have the same type 
and only the connectivity matters (e.g. in studies of node failure [8, 20]). 

The remainder of this section is dedicated to illustrating some of these applica¬ 
tions by extending our primitive model of the eye. 

Example 5.1 (Adding behavior). One typical task in constructing a model or sim¬ 
ulation of a subject is to add some dynamics or behavior to an existing system. 
Recall our simple model of the eye from Example 3.6. It is clearly incomplete— 
given time, we might want to model ganglion cells that summarize the behavior of 
several retinal nerve cells, the special dynamics of detector cells (where "activating" 
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a rod or cone actually means inhibiting its spontaneous rate of fire), and the many 
projections from retinal nerves to other nuclei in the cortex. 

For now, we will do something relatively simple: add the ability to "blink". That 
is, we need to change the mode set of our eyeball E from {*} to something like 
{open, shut}, corresponding to the two wiring diagrams below 



m—open m=closed 


It's possible to specify the dynamics for open/shut directly inside E; one simply 
defines the state set of E to be 

Se = {open, shut} x x Sn 2 x ... x Sy*. 

and the state update function /{ n to be a similar product 

An _ An fin fin fin 

JE /open/shut /N \ JN 2 **’ /N/t 

where /<^, en / s h u t ■ —> {open/shut} maps to closed if a sufficient number of neu¬ 

rons are depolarized, and to open otherwise. 

For various reasons, this way of specifying the dynamics may not be preferable. 
For one, it is not biological: the eye offloads this sort of computation to other areas 
of the brain. Nor is it causal; reflexes do not trigger on stored states but on active 
signals. Finally, it is not modular: we would like to be able to modify the dynamics 
of a system directly by rewiring the diagram—i.e. changing the topology—but in 
this case we would have to change the state update function directly. 

As an alternative, we can outsource the computation of /^ en / shut to another 
modal box P (here, P stands for pons, an area of the brainstem that mediates the 
blink reflex). We define Sp = {open, shut} and f'p : {0,1}” — > Sp to map to closed 
if the average of the signals is above a certain threshold, and to open otherwise. 
Mp = {*}. See Figure 5. Note that we need to add a new input port to box E to 
receive an open or shut signal from P. 

For later use, we define this entire system as a mode-dependent network Blink : 
(£], E 2 , P) —> B, where B is the containing box. 

Example 5.2 (Neural networks). In (artificial) neural networks, we use neurons to 
form interconnected layers (there is no connection within a layer). Each neural net¬ 
work can be thought of as a classifier, e.g. a {0, l}-valued function of the inputs: 
nodes in the first layer represent features, nodes in the second layer interpret these 
features into new features, and so on for additional layers. We can model such neural 
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Figure 5: Two eyes are linked to a modal box P that controls blinking. 


networks as mode-dependent networks and connect them to other mode-dependent 
networks such as Example 5.1. For simplicity, we only consider feedforward net¬ 
works, e.g. networks of neurons without loops. 

A (cortical) neuron is a mode-dependent network 

Neuron = (e,a\) : ((M N/ N), (M s ,S)) -4- (Mr,R) 


with a nucleus (Mn, N), a summation box S = (Ms, S), and a wrapper R = (Mr, R). 
Both S and R are mode-independent; Mr = {polarized, depolarized, hyperpolar- 
ized}, as in Example 4.3. The nucleus is incapable of receiving signals when it is 
either depolarized (firing), or hyperpolarized (just finished firing). 


</>:(N("*n),S(*))-KR(*) 


N ta :{0,l} 



N° ut :{0,l} 


m;v=polarized 


cp:(N(mn),S(*))-^R(*) 





N° ut :{0,l} 


m;v;=depolarized / hyperpolarized 


To define a modal dynamical system on top of Neuron = (e, C\), we provide the 
following data: 

State sets. Let k = |S m | be the number of inputs to S. Then S ( y = Mr = {polarized, 
depolarized, hyperpolarized} and S$ = 1R + x Wg, where Wg = 1R ,C denotes a 
space of weight vectors. As the container, R has state set Sr = x S$- 

Underlying mode function. Cj$ '■ $S ~t Ms is trivial, since the mode set of S is 
trivial. Similarly with q&. : Sn —> Mr is just the identity. 
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State update function. Define a firing threshold a E [0,1]. The state update func¬ 
tion for f m : Sn x {0,1} ,C —» S,v is given by 

k 

fg l (s,W 1 ,...,W k ,X 1 / ...,X k ) = (Y^WiXi,W lr ...,W k ) 

i 

where w E Wg. When the mode N is polarized. 


/nOwO 


depolarized x > a 
polarized otherwise 


When the mode of N is depolarized or hyperpolarized. 



hyperpolarized m = depolarized 
polarized m = hyperpolarized 


Readout. Finally, the readout of S is just fg ut (s, w \,..., w k ) = s, while the readout of 
N is 


fT(m) 


1 m = polarized 
0 otherwise 


The readout function for the wrapper R is just the product fg ut x /", ut . 

Given a modal dynamical system on a neuron, we can form new modal dynam¬ 
ical systems by forming products and compositions of various copies of neuron, o~t ). 
A layer of neurons is a mode-dependent network 


V/ = (e,cr\) : (Neuron ® ... <g> Neuron) —> Vj 


formed by a tensor product of neurons, with the condition that there are no connec¬ 
tions between neurons inside a layer. 



Just as in Figure 4, the diagram above depicts a composition of the mode-dependent 
network Vi : (R k ,R 4 ) —> V\ with several copies of Neuron : (S, —>■ R 
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to form a new mode-dependent network, Vi o (Neuron,Neuron). Again, the 
composition is what allows us to "zoom in" on the structure of each individual 
neuron. As before, the thicker arrows denote several parallel wires. The dynamics 
of this system are given directly from the coherence maps in Equation 8 . 



Figure 6 : A feedforward network composed of three layers, written as a mode- 
dependent network V = (e, cy) : (V), V 2 , V 3 ) —* V in ^mdn- Note that the diagram 
pictured above is technically not a legal wiring diagram: typically multiple inputs cannot 
map into the same port, but we have used a shorthand here to represent the many 
input ports to each layer. 


It is now possible to take the tensor product of layers to form wider, parallel 
layers, or to wire multiple layers together to form a multilayer neural network as in 
Figure 6 . This is done by defining a morphism V = (e,cr\) : (V ], Vi, V 3 ) —> V from 
each box V, to a wrapper V, and the process may be continued ad infinitum. 

In principle it should not be hard to prove that there is an additional operad 
of feedforward networks, denoted <^ffn, as a suboperad of ^mdi\i, with objects ten¬ 
sor products of neurons and morphisms feedforward networks. Affn constitutes a 
domain-specific language for writing neural networks, one that allows them to be 
easily composed in serial, in parallel, or by insertion. 

Example 5.3 (Plug and play). As promised, we can now "plug in" Example 3.7, 
Example 5.1, and Example 5.2 to form an extended, mode-dependent model of the 
visual system, VS = (e,cr) : (B,V) —> VS. Essentially: light comes in through 
the eyes and activates the retinal nerve cells, whose signals are ( 1 ) used to define a 
blink reflex through the pons and ( 2 ) fed to a slice of the visual cortex, where the 
information is interpreted, analyzed, and fed as a { 0 , 1 } output to some other region 
of the brain. 

The entire system is reducible to the retinal nerves and cortical neurons. Recall 
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Figure 7: A wiring diagram of the visual pathway. 


the mode-dependent networks defined up to now: 


Nerve 

(M n ,N) -4 R 

Eye 

(Ri,..., R n ) — 4 E 

Blink 

(E lr E 2r P)^B 

Neuron 

((M n ,N),S) -4 

V/ 

(Ri,..., R n ) —>■ V{ 

V 

(V,v 2 ,v 3 ) -4 v 

vs 

(B,V) -4 VS 


Then the previous statement, that the system is reducible to nerves and neurons, is 
just the statement of the composition: 

VSo (V,Blink) o (Eye,Eye,Vi, V 2 ,V 3 ) o (Nerve,...,Nerve,Neuron,...,Neuron). 
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